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DETAILED ACTION 



Specification 

1 . The disclosure is objected to because of the following informalities: 

On page 4, line 1 1, the name "Canagarajar" is misspelled according to the spelling on 
article "A Robust Automatic Clustering Scheme for Image Segmentation Using Wavelets", 
provided with this office action. The correct spelling is "Canagarajah". 
Appropriate correction is required. 



Claim Rejections - 35 USC § 102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

3. Claims 1, 2, 9, 1 1, 13, 15, 16, 21-23, 30-34, 37, 38, 40, 42, 44-46 and 49 rejected under 
35 U.S.C. 102(b) as being anticipated by US 2002/0191861 to Cheatle. 



As to claim 1, Cheatle discloses an automatic image cropping system for use with a 
portable device having an image capture mechanism and a limited resource for storing or 
transmitting captured information (paragraph 0080 and 0081, wherein the camera corresponds 
to a portable device and the memory corresponds to a limited resource for storing), the system 
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comprising a region of interest suggestion engine defining plural image region candidates by 
performing image segmentation on an image stored in digital form (Fig. 10, elements 202-214 
and paragraphs 103, 130 and 133, wherein the features of interest are determined and 
alternative constraint set combination are performed on the image, which corresponds to image 
segmentation, to identify image cropping candidates each of which corresponds to a image 
region candidate), determining if an image region candidate is likely to be more or less 
interesting to a user than another image region candidate (Fig. 10, elements 202-214, and 
paragraph 154 and 155, wherein the penalty score is determined for each crop candidate, which 
corresponds to an image region candidate, and the lower the penalty the more likely the image 
region candidate is of interest to the user), and selecting an image region candidate determined 
as likely to be of most interest to the user (Fig. 10, elements 214 and paragraph 0154 and 0155, 
wherein the candidate region with the lowest penalty, which corresponds to the image 
candidate region likely to be of most interest to the user, is selected). 

As to claim 2, Cheatle discloses the system of claim 1, wherein said region of interest 
suggestion engine measures entropies of the image region candidates and uses entropy thus 
measured as a measure of likelihood of user interest (paragraph 0154 and 0155, wherein 
penalty score corresponds to entropies). 

As to claim 9, Cheatle discloses the system of claim 1, wherein said region of interest 
suggestion engine segments the image based on image texture and color consistency 
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(paragraph 128, wherein texture and color consistency are evaluated to determine the features 
for use with the region of interest suggestion engine). 

As to claim 11, Cheatle discloses the system of claim 1, wherein said region of interest 
suggestion engine employs a fuzzy k-mean clustering method to perform the image 
segmentation (paragraph 95). 

As to claim 13, Cheatle discloses the system of claim 1, wherein said region of interest 
suggestion engine performs color transformation on an image stored in digital form (paragraph 
86, wherein color transformation is performed for example to a YCC color space format). 

As to claim 15, Cheatle discloses the system of claim 1, wherein said region of interest 
suggestion engine measures sizes of image region candidates relative to a common viewing 
area of the image and uses relative size thus measured as a measure of likelihood of user 
interest (paragraph 133, wherein constraint set include using aspect ratio and size constraints 
which corresponds to using relative size thus measured as a measure of likelihood of user 
interest). 

As to claim 16, Cheatle discloses the system of claim 1, wherein said region of interest 
suggestion engine measures locations of image region candidates relative to a common 
viewing area of the image and uses relative location thus measured as a measure of likelihood 



Application/Control Number: 10/567,499 Page 5 

Art Unit: 2624 

of user interest (paragraphs 133-137 and 140, wherein penalties correspond to likelihood and 
they are based in part on candidate region location). 

As to claim 21, Cheatle discloses the system of claim 1, wherein said engine uses camera 
sensor data to determine likelihood of user interest (paragraph 133 and 140, wherein pixel data 
corresponds to camera sensor data and pixel data is used to determine likelihood). 

As to claim 22, Cheatle discloses an automatic image cropping method, comprising: 

Performing image segmentation on an image stored in digital form thereby defining 
plural image region candidates (Fig. 10, elements 202-214 and paragraphs 103, 130 and 133, 
wherein the features of interest are determined and alternative constraint set combination are 
performed on the image, which corresponds to image segmentation, to identify image cropping 
candidates each of which corresponds to a image region candidate); 

Determining if an image region candidate is likely to be more or less interesting to a user 
than another image region candidate (Fig, 10, elements 202-214, and paragraph 154 and 155, 
wherein the penalty score is determined for each crop candidate, which corresponds to an 
image region candidate, and the lower the penalty the more likely the image region candidate is 
of interest to the user); and 

Selecting an image region candidate determined as likely to be of most interest to the user 
(Fig. 10, elements 214 and paragraph 0154 and 0155, wherein the candidate region with the 
lowest penalty, which corresponds to the image candidate region likely to be of most interest to 
the user, is selected). 
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As to claim 23, please refer to the rejection made for claim 2 above. 

As to claim 30, Cheatle discloses the method of claim 22, further comprising suggesting 
the selected image region candidate to a user (Fig. 10, element 214 and paragraph 0155 and 
0158, wherein the selection of the candidate to a user is done in 214 and may automatically). 

As to claim 31, Cheatle discloses the method of claim 30, further comprising receiving a 
user confirmation of the selected image region candidate (paragraph 0156, wherein the user 
may confirm the candidate or suggest improvements). 

As to claim 32, Cheatle discloses the method of claim 31, further comprising processing 
the image based on the user confirmation (paragraph 0156 wherein the user may suggest 
improvements). 

As to claim 33, Cheatle discloses the method of claim 31, further comprising segregating 
the selected image region candidate from at least one other part of the image in response to 
receipt of the user confirmation (paragraph 0156, wherein the user may suggest improvements 
such as more or less room in the crop limits which correspond to segregating the a part of the 
image from another). 
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As to claim 34, Cheatle discloses the method of claim 31, further comprising saving the 
selected image region candidate absent image contents external to the selected image region in 
response to receipt of the user confirmation (paragraphs 80, 81 and 156, wherein it is inherent 
that the region inside the selected crop limits by the user are saved the memory). 

As to claim 37, Cheatle discloses the method of claim 30, further comprising: 
Receiving a user contradiction of the selected image region candidate; and 
Selecting a new image region candidate determined as most likely to be of most interest 
to the user based on the user contradiction (paragraph 156, wherein the user may reject a 
candidate which corresponds to user contradiction and a new candidate may be evaluated using 
a new crop limit). 

As to claim 38, please refer to the rejection made for claim 9 above. 
As to claim 40, please refer to the rejection made for claim 1 1 above. 
As to claim 42, please refer to the rejection made for claim 13 above. 
As to claim 44, please refer to the rejection made for claim 15 above. 
As to claim 45, please refer to the rejection made for claim 16 above. 

As to claim 46, Cheatle discloses the method of claim 22, further comprising capturing an 
image in digital form (paragraph 86). 



As to claim 49, please refer to the rejection made for claim 21 above. 
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Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

5. Claims 10 and 39 are rejected under 35 U.S.C. 103(a) as being xmpatentable over Cheatle 
in view of an article entitled "A Robust Automatic Clustering Scheme for Image Segmentation 
Using Wavelets" by Porter et al. ("Porter"). 

As to claim 10, Cheatle discloses the system of claim 9. 

Cheatle does not disclose expressly wherein said region of interest suggestion engine 
uses vectors calculated from Wavelet transform to represent texture information. 

Porter discloses a region of interest suggestion engine that uses vectors calculated from 
Wavelet transform to represent texture information (Porter, Abstract). 

Cheatle & Porter are combinable because they are from the same art of image 
processing and more specifically image segmentation. 

At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to use the Wavelet transform to represent texture information, as taught by Porter, in the 
automatic image cropping system disclosed by Cheatle. 

The suggestion/motivation for doing so would have been that using the Wavelet 
transform provides a more robust image segmentation (Porter, Abstract). 
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Therefore, it would have been obvious to combine Cheatle with Porter to obtain the 
invention as specified in claim 10. 

As to claim 39, please refer to the rejection of claim 10 above. 

6. Claims 14 and 43 are rejected under 35 U.S.C. 103(a) as being unpatentable over Cheatle 
further in view of USPN 5,341,226 to Shiau. 

As to claim 14, Cheatle discloses the system of claim 13. 

Cheatle does not disclose expressly wherein said region of interest suggestion engine 
transforms an image in RGB format into HUV (Hue, Saturation, and Intensity) format. 

Shiau discloses transforming an image in RGB format into HUV (Hue, Saturation, and 
Intensity) format (column 5, line 64 - column 6, line 23 wherein HSI corresponds to HUV). 

Cheatle & Shiau are combinable because they are from the same art of image 
processing. 

At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to transform the image region from RGB to HUV as taught by Shiau in the automatic 
cropping system disclosed by Cheatle. 

The suggestion/motivation for doing so would have been to represent intensity as one 
signal (Shiau, column 5, line 68 - column 6, line 2). 

Therefore, it would have been obvious to combine Cheatle with Shiau to obtain the 
invention as specified in claim 14. 
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As to claim 43, please refer to the rejection of claim 14 above. 

* 

7. Claims 17 and 47 are rejected under 35 U.S.C. 103(a) as being unpatentable over Cheatle 
in view of US Patent Publication 2003/0035580 to Wang et al. ("Wang"). 

As to claim 1 7, Cheatle discloses the system of claim 1 . 

Cheatle does not disclose expressly wherein the region of interest suggestion engine 
pre-processes the image to eliminate noise in blurred text histograms to smooth the image. 

Wang discloses image segmentation comprising of pre-processes the image to eliminate 
noise in blurred text histograms to smooth the image (paragraph 0040 and 0041). 

Cheatle & Wang are combinable because they are from the same art of image 
processing. 

At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to use the pre-processing technique of the image to eliminate noise as taught by Wang in 
the automatic cropping system disclosed by Cheatle. 

The suggestion/motivation for doing so would have been to reduce the salt and pepper 
noise in a scene image (Wang, paragraph 0041). 

Therefore, it would have been obvious to combine Cheatle with Wang to obtain the 
invention as specified in claim 17. 

As to claim 47, please refer to the rejection of claim 1 7 above. 
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8. Claim 35 is rejected under 35 U.S.C. 103(a) as being unpatentable over Cheatle in view 
of US Patent Application Publication 2003/0122942 to Parker et al. ("Parker"). 

As to claim 35, Cheatle discloses the method of claim 31. 

Cheatle does not disclose expressly further comprising transmitting the selected image 
region candidate absent image contents extemal to the selected image region in response to 
receipt of the user confirmation. 

Parker discloses transmitting the selected image region candidate absent image contents 
extemal to the selected image region (paragraph 0027). 

Cheatle & Parker are combinable because they are from the same art of image 
processing. 

At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to transmit the region selected by the user using the network connection disclosed by 
Parker. 

The suggestion/motivation for doing so would have been to transmit a cropped image. 
Therefore, it would have been obvious to combine Cheatle vdth Parker to obtain the 
invention as specified in claim 35. 

9. Claim 36 is rejected under 35 U.S.C. 103(a) as being unpatentable over Cheatle in view 
of US Patent Application Publication 2002/01 14535 to Luo. 
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As to claim 36, Cheatle discloses the method of claim 31 . 

Cheatle does not disclose expressly zooming in on the im^gc region candidate in 
response to receipt of the user confirmation. 

Luo discloses zooming in on the image region candidate (paragraph 0022). 

Cheatle & Luo are combinable because they are from the same art of image processing. 

At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to zoom in on the image region candidate, as taught by Luo, in response to user 
confirmation as disclosed by Cheatle. 

The suggestion/motivation for doing so would have been to improve the quality of 
finished photographs and picture sharing experiences (Luo, paragraph 0002). 

Therefore, it would have been obvious to combine Cheatle with Luo to obtain the 
invention as specified in claim 36. 

Allowable Subject Matter 

10. Claims 3-8, 12, 18-20, 24-29, 41 and 48 are objected to as being dependent upon a 
rejected base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 



Conclusion 

1 1 . The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

USPN 6,654,507 to Luo discloses an image segmentation and cropping method. 
USPN 6,654,506 to Luo et al. discloses an image segmentation and cropping method. 
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US 2003/0122942 to Parker et al. discloses an image segmentation and cropping method. 
US 6,545,743 to Luo et ai. discloses an image segmentation and cropping method. 
US 2003/0059121 to Savakis et al. discloses an image segmentation and cropping 
method. 

US 2003/0052962 to Wilk discloses an image segmentation and cropping method. 
US 2003/0044078 to Joshi et al. discloses an image segmentation and cropping method. 
US 2002/0131641 to Luo et al. discloses an image segmentation and cropping method. 
US 2002/01 1 1 1 88 to Harma et al. discloses an image segmentation and cropping method. 
US 2002/0093670 to Luo et al. discloses an image segmentation and cropping method. 
USPN 6,282,317 to Luo et al. discloses an image segmentation and cropping method. 
USPN 6,256,414 to Mancuso et al. discloses an image segmentation and cropping 
method. 

USPN 5,341,226 to Shiau discloses an image segmentation and cropping method. 

1 2. Any inquiry conceming this communication or earlier communications from the 
examiner should be directed to Aaron W. Carter whose telephone number is (571) 272-7445. 
The examiner can normally be reached on 8am - 4:30 am (Mon. - Fri.). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on (571) 272-7429. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for impublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 




Aaron Carter 
AU 2624 



